Assessment of the learning curve in health technologies: a systematic review

Objective: We reviewed and appraised the methods by which the issue of the learning curve has been addressed during health technology assessment in the past. Method: We performed a systematic review of papers in clinical databases (BIOSIS, CINAHL, Cochrane Library, EMBASE, HealthSTAR, MEDLINE, Science Citation Index, and Social Science Citation Index) using the search term "learning curve:" Results: The clinical search retrieved 4,571 abstracts for assessment, of which 559 (12%) published articles were eligible for review. Of these, 272 were judged to have formally assessed a learning curve. The procedures assessed were minimal access (51%), other surgical (41%), and diagnostic (8%). The majority of the studies were case series (95%). Some 47% of studies addressed only individual operator performance and 52% addressed institutional performance. The data were collected prospectively in 40%, retrospectively in 26%, and the method was unclear for 31%. The statistical methods used were simple graphs (44%), splitting the data chronologically and performing a t test or chi-squared test (60%), curve fitting (12%), and other model fitting (5%). Conclusions: Learning curves are rarely considered formally in health technology assessment. Where they are, the reporting of the studies and the statistical methods used are weak. As a minimum, reporting of learning should include the number and experience of the operators and a detailed description of data collection. Improved statistical methods would enhance the assessment of health technologies that require learning

Assessing the learning curve effect in health technologies: Lessons from the non-clinical literature

Introduction: Many health technologies exhibit some form of learning effect, and this represents a barrier to rigorous assessment. It has been shown that the statistical methods used are relatively crude. Methods to describe learning curves in fields outside medicine, for example, psychology and engineering, may be better. Methods: To systematically search non–health technology assessment literature (for example, PsycLit and Econlit databases) to identify novel statistical techniques applied to learning curves. Results: The search retrieved 9,431 abstracts for assessment, of which 18 used a statistical technique for analyzing learning effects that had not previously been identified in the clinical literature. The newly identified methods were combined with those previously used in health technology assessment, and categorized into four groups of increasing complexity: a) exploratory data analysis; b) simple data analysis; c) complex data analysis; and d) generic methods. All the complex structured data techniques for analyzing learning effects were identified in the nonclinical literature, and these emphasized the importance of estimating intra- and interindividual learning effects. Conclusion: A good dividend of more sophisticated methods was obtained by searching in nonclinical fields. These methods now require formal testing on health technology data sets

A phase field method for tomographic reconstruction from limited data.

Classical tomographic reconstruction methods fail for problems in which there is extreme temporal and spatial sparsity in the measured data. Reconstruction of coronal mass ejections (CMEs), a space weather phenomenon with potential negative effects on the Earth, is one such problem. However, the topological complexity of CMEs renders recent limited data reconstruction methods inapplicable. We propose an energy function, based on a phase field level set framework, for the joint segmentation and tomographic reconstruction of CMEs from measurements acquired by coronagraphs, a type of solar telescope. Our phase field model deals easily with complex topologies, and is more robust than classical methods when the data are very sparse. We use a fast variational algorithm that combines the finite element method with a trust region variant of Newton’s method to minimize the energy. We compare the results obtained with our model to classical regularized tomography for synthetic CME-like images

REMCARE : pragmatic multi-centre randomised trial of reminiscence groups for people with dementia and their family carers : effectiveness and economic analysis

Background Joint reminiscence groups, involving people with dementia and family carers together, are popular, but the evidence-base is limited. This study aimed to assess the effectiveness and cost-effectiveness of joint reminiscence groups as compared to usual care. Methods This multi-centre, pragmatic randomised controlled trial had two parallel arms: intervention group and usual-care control group. A restricted dynamic method of randomisation was used, with an overall allocation ratio of 1:1, restricted to ensure viable sized intervention groups. Assessments, blind to treatment allocation, were carried out at baseline, three months and ten months (primary end-point), usually in the person's home. Participants were recruited in eight centres, mainly through NHS Memory Clinics and NHS community mental health teams. Included participants were community resident people with mild to moderate dementia (DSM-IV), who had a relative or other care-giver in regular contact, to act as informant and willing and able to participate in intervention. 71% carers were spouses. 488 people with dementia (mean age 77.5) were randomised: 268 intervention, 220 control; 350 dyads completed the study (206 intervention, 144 control). The intervention evaluated was joint reminiscence groups (with up to 12 dyads) weekly for twelve weeks; monthly maintenance sessions for further seven months. Sessions followed a published treatment manual and were held in a variety of community settings. Two trained facilitators in each centre were supported by volunteers. Primary outcome measures were self-reported quality of life for the person with dementia (QoL-AD), psychological distress for the carer (General Health Questionnaire, GHQ-28). Secondary outcome measures included: autobiographical memory and activities of daily living for the person with dementia; carer stress for the carer; mood, relationship quality and service use and costs for both. Results The intention to treat analysis (ANCOVA) identified no differences in outcome between the intervention and control conditions on primary or secondary outcomes (self-reported QoL-AD mean difference 0.07 (-1.21 to 1.35), F = 0.48, p = 0.53). Carers of people with dementia allocated to the reminiscence intervention reported a significant increase in anxiety on a General Health Questionnaire-28 sub-scale at the ten month end-point (mean difference 1.25 (0.25 to 2.26), F = 8.28, p = 0.04). Compliance analyses suggested improved autobiographical memory, quality of life and relationship quality for people with dementia attending more reminiscence sessions, however carers attending more groups showed increased care-giving stress. Economic analyses from a public sector perspective indicated that joint reminiscence groups are unlikely to be cost-effective. There were no significant adverse effects attributed to the intervention. Potential limitations of the study include less than optimal attendance at the group sessions—only 57% of participants attended at least half of the intervention sessions over the 10 month period, and a higher rate of study withdrawal in the control group. Conclusions This trial does not support the clinical effectiveness or cost-effectiveness of joint reminiscence groups. Possible beneficial effects for people with dementia who attend sessions as planned are offset by raised anxiety and stress in their carers. The reasons for these discrepant outcomes need to be explored further, and may necessitate reappraisal of the movement towards joint interventions

What can we learn from trial decliners about improving recruitment? Qualitative study

Background Trials increasingly experience problems in recruiting participants. Understanding the causes of poor recruitment is critical to developing solutions. We interviewed people who had declined a trial of an innovative psychological therapy for depression (REFRAMED) about their response to the trial invitation, in order to understand their decision and identify ways to improve recruitment. Methods Of 214 people who declined the trial, 35 (16 %) gave permission to be contacted about a qualitative study to explore their decision. Analysis of transcripts of semi-structured interviews was informed by grounded theory. Results We interviewed 20 informants: 14 women and six men, aged 18 to 77 years. Many interviewees had prior experience of research participation and positive views of the trial. Interviewees’ decision making resembled a four-stage sequential process; in each stage they either decided not to participate in the trial or progressed to the next stage. In stage 1, interviewees assessed the invitation in the context of their experiences and attitudes; we term those who opted out at this stage ‘prior decliners’ as they had an established position of declining trials. In stage 2, interviewees assessed their own eligibility; those who judged themselves ineligible and opted out at this stage are termed ‘self-excluders’. In stage 3, interviewees assessed their need for the trial therapy and potential to benefit; we term those who decided they did not need the trial therapy and opted out at this stage ‘treatment decliners’. In stage 4, interviewees deliberated the benefits and costs of trial participation; those who opted out after judging that disadvantages outweighed advantages are termed ‘trial decliners’. Across all stages, most individuals declined because they judged themselves ineligible or not in need of the trial therapy. While ‘prior decliners’ are unlikely to respond to any trial recruitment initiative, the factors leading others to decline are amenable to amelioration as they do not arise from a rejection of trials or a personal stance. Conclusions To improve recruitment in similar trials, the most successful interventions are likely to address patients’ assessments of their eligibility and their potential to benefit from the trial treatment, rather than reducing trial burden

A phase field method for tomographic reconstruction from limited data

Classical tomographic reconstruction methods fail for problems in which there is extreme temporal and spatial sparsity in the measured data. Reconstruction of coronal mass ejections (CMEs), a space weather phenomenon with potential negative effects on the Earth, is one such problem. However, the topological complexity of CMEs renders recent limited data reconstruction methods inapplicable. We propose an energy function, based on a phase field level set framework, for the joint segmentation and tomographic reconstruction of CMEs from measurements acquired by coronagraphs, a type of solar telescope. Our phase field model deals easily with complex topologies, and is more robust than classical methods when the data are very sparse. We use a fast variational algorithm that combines the finite element method with a trust region variant of Newton’s method to minimize the energy. We compare the results obtained with our model to classical regularized tomography for synthetic CME-like images

Individual cognitive stimulation therapy for dementia : a clinical effectiveness and cost-effectiveness pragmatic, multicentre, randomised controlled trial

Background Group cognitive stimulation therapy programmes can benefit cognition and quality of life for people with dementia. Evidence for home-based, carer-led cognitive stimulation interventions is limited. Objectives To evaluate the clinical effectiveness and cost-effectiveness of carer-delivered individual cognitive stimulation therapy (iCST) for people with dementia and their family carers, compared with treatment as usual (TAU). Design A multicentre, single-blind, randomised controlled trial assessing clinical effectiveness and cost-effectiveness. Assessments were at baseline, 13 weeks and 26 weeks (primary end point). Setting Participants were recruited through Memory Clinics and Community Mental Health Teams for older people. Participants A total of 356 caregiving dyads were recruited and 273 completed the trial. Intervention iCST consisted of structured cognitive stimulation sessions for people with dementia, completed up to three times weekly over 25 weeks. Family carers were supported to deliver the sessions at home. Main outcome measures Primary outcomes for the person with dementia were cognition and quality of life. Secondary outcomes included behavioural and psychological symptoms, activities of daily living, depressive symptoms and relationship quality. The primary outcome for the family carers was mental/physical health (Short Form questionnaire-12 items). Health-related quality of life (European Quality of Life-5 Dimensions), mood symptoms, resilience and relationship quality comprised the secondary outcomes. Costs were estimated from health and social care and societal perspectives. Results There were no differences in any of the primary outcomes for people with dementia between intervention and TAU [cognition: mean difference –0.55, 95% confidence interval (CI) –2.00 to 0.90; p-value = 0.45; self-reported quality of life: mean difference –0.02, 95% CI –1.22 to 0.82; p-value = 0.97 at the 6-month follow-up]. iCST did not improve mental/physical health for carers. People with dementia in the iCST group experienced better relationship quality with their carer, but there was no evidence that iCST improved their activities of daily living, depression or behavioural and psychological symptoms. iCST seemed to improve health-related quality of life for carers but did not benefit carers’ resilience or their relationship quality with their relative. Carers conducting more sessions had fewer depressive symptoms. Qualitative data suggested that people with dementia and their carers experienced better communication owing to iCST. Adjusted mean costs were not significantly different between the groups. From the societal perspective, both health gains and cost savings were observed. Conclusions iCST did not improve cognition or quality of life for people with dementia, or carers’ physical and mental health. Costs of the intervention were offset by some reductions in social care and other services. Although there was some evidence of improvement in terms of the caregiving relationship and carers’ health-related quality of life, iCST does not appear to deliver clinical benefits for cognition and quality of life for people with dementia. Most people received fewer than the recommended number of iCST sessions. Further research is needed to ascertain the clinical effectiveness of carer-led cognitive stimulation interventions for people with dementia

Barriers and facilitators to change in the organisation and delivery of endoscopy services in England and Wales: a focus group study

Objective: Explore professional views of changes to gastroenterology service organisation and delivery and barriers and facilitators impacting on change. The work was undertaken as part of an evaluation in endoscopy service provision catalysed by the Modernising Endoscopy Services Programme of the Modernisation Agency. Design: Focus groups followed by analysis and group-working activities identifying key themes. Setting: English and Welsh secondary care gastroenterology units. Participants: 20 professionals working in gastroenterology in England and Wales. Medical, surgical and nursing specialists including endoscopy nurses. Opportunistic sampling to include senior people in leadership and management roles who were directly involved in service modernisation, excluding those involved in the Modernisation Endoscopy Services Programme. Results: Four 1.5 h focus groups took place in 2007. Summative and thematic analyses captured essential aspects of text and achieved consensus on key themes. 4 themes were revealed: 'loss of personal autonomy and erosion of professionalism', 'lack of senior management understanding', 'barriers and facilitators to change' and 'differences between English and Welsh units'. Themes indicated that low staff morale, lack of funding and senior management support were barriers to effective change. Limitations to the study include the disproportionately low number of focus group attendees from English units and the time delay in reporting these findings. Conclusions: Despite ambitions to implement change, ineffective management support continued to hamper modernisation of service organisation and delivery. While the National Health Service Modernisation Agency Modernising Endoscopy Services Programme acted as a catalyst for change, affecting the way staff work, communicate and think, it was not effective in heralding change itself. However, gastroenterologists were keen to consider the potential for change and future service modernisation. The methodological framework of innovative qualitative enquiry offers comprehensive and rigorous enhancement of quantitative studies, including randomised trials, when a mixed methods approach is needed.7 page(s